Complexity Control of Image Processing Network Architectures through Regularization

The complexity of a Neural Network (NN) architecture has long been identified as of crucial importance for the NN overall generalization capability: a network which is too simple or too complex will generalize poorly, while having performances on learning set poor for the former, or close-to-perfect for the latter. For applications in image processing, the problem is usually particularly accute, since even an image of moderate size -say 16x16 pixelswill lead to a NN architecture with a potentially huge first layer: hence calling for particular care to control the complexity.